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Object tracking is one of the challenging issues in computer vision and video 
processing, which has several potential applications. In this paper, initially, a 
moving object is selected by frame differencing method and extracted the 
object by segment thresholding. The bubble sort algorithm (BSA) arranges 
the regions (large to small) to make sure that there is at least one big region 
(object) in object detection process. To track the object, a motion model is 
constructed to set the system models of Alpha-Beta (a-f) filter, Linear 
Kalman filter (LKF) and Extended Kalman filter (EKF). Many experiments 
have been conducted on balls with different sizes in image sequences and 
compared their tracking performance in normal light and bad light 
conditions. The parameters obtained are the root mean square error (RMSE), 
absolute error (AE), object tracking error (OTE), Tracking detection rate 
(TDR), and peak signal-to-noise ratio (PSNR) and they are compared to find 
the algorithm that performs the best for two conditions. 
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1. INTRODUCTION 

Computer vision is the advancement in science and technology that include methods for acquiring, 
processing, analyzing, and understanding images. As a scientific discipline, computer vision is fretful with 
the theory behind artificial systems that excerpt information from images. The image data can take various 
forms, such as video sequences, views from many cameras, or multi-dimensional data from a medical 
scanner. As a scientific discipline, computer vision pursues to relate its theories and models to the design of 
computer vision systems. Many research extends to include: uses in security and surveillance, video 
communication and compression, Navigation, display technology, High-Level Video Analysis, traffic 
control, Metrology, video editing, augmented reality and Human-Computer interfaces to medical 
imaging [1-2]. Given, the most important state (e.g., position and velocity) of a target object in the leading 
image, the objective of tracking is to estimate the states of the target in the subsequent frames. Even though 
object tracking has been studied for more than a few decades and significant evolution has been made in 
recent years [3-7], it remains a challenging problem. 

Object illustration is one of the key works in any pictorial tracking algorithm, and many outlines 
have been proposed [8]. Since the primary work of Lucas and Kanade [9], based on the raw concentration of 
values have been broadly used for tracking [10]. The Lucas and Kanade approach [11], do not take great 
occurrence changeability into account and thus, do not succeed due to the dissimilarity of the chromatic 
properties of a target object. Matthews et al. [12] established a template update method by exploiting the 
information of the first frame to accurate points. Towards better account for changes in the presence of the 
object, subspace-based tracking approach [7,13] have been proposed. In [14], Hager and Belhumeur 
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recommended a proficient Lucas and Kanade algorithm and used low dimensional designs for tracking 
changing illumination conditions. Several techniques have been anticipated for video object segmentation. 
Commonly, these methods can be approximately classified into two types [15, 16]: background structure 
based video object segmentation and foreground extraction based video object segmentation. 

In recent times, a number of tracking methods based on sparse representations have been proposed 
[17-21]. To expand tracking robustness, a local sparse presence model was proposed in [22] using the mean 
shift algorithm to detect objects. By the presumptuous representation of particles as together, H. Lu et al [17] 
expressed object tracking as a multi-task sparse learning problem. J. Fiscus, J. Garofolo, T. Rose, and M. 
Michel, [23] provides an experimental evaluation of several tracking algorithms on the advanced video and 
signal based surveillance, in a conference on advanced video and signal based surveillance, using dataset for 
following various people. The effort of such reviews is of better-quality still, as in [24] which discuss 
tracking specific targets only, for example, sports players. The survey of J.C. McCall and M. M. Trivedi [25] 
is on tracking paths for driver aid. 

In this paper, initially, a moving object is selected by frame differencing method and extracted 
object by segment thresholding. The bubble sort algorithm (BSA) arranges the regions (large to small) to 
make sure that there is at least one big region (object). To track the object, a motion model is constructed to 
set the system models of the a-f filter, LKF and EKF. Many experiments have been conducted on balls with 
different sizes in image sequences and compared their tracking performance in normal light and bad light 
conditions. The parameters obtained are RMSE, AE, OTE, TDR, and peak signal-to-noise ratio (PSNR). 
Experimental results indicate that the proposed algorithms perform well for the detection and tracking of 
different sizes of balls in video sequences captured by using a static iphone6 camera. The organization of the 
paper is as follows. The moving object detection and tracking methods are proposed in Section 2. Section 3 
presents experimental results and discussions. Finally, the conclusion is given in Section 4. 


2. THE PROPOSED METHOD 
2.1. Simple Background Subtraction 

Simple Background Subtraction (SBS) is the base for several developments in object detection 
[26-27] which detects moving objects through calculating the absolute difference between the reference 
image C(x, y) [28] and incoming video frame I; (x,y). Notice that the reference image C(x, y) must have 
only static background; the incoming frame features possible moving objects along with background 
information. After the reference image C(x,y) and the incoming video frame (x,y) are taken from a 
video sequence, the detected binary motion detection mask M (x, y) is calculated as follows: 


LEA Vy = COG Le 


0; if Gy) -C(xy)| St (1) 


M(x,y) =| 


where, T is an empirically a selection of threshold, which is used to distinguish pixels of moving objects from 
those of the background in an image frame. The existence of moving objects is indicated if the absolute 
difference between the reference image C(x,y) and the incoming video frame [;,(x,y) go beyond t these 
pixels of the detection mask M(x, y) are then labeled with 1 and those that are non-active are labeled with 0. 


2.2. Bubble Sort Algorithm 

The bubble sort algorithm is based on a comparison of consecutive elements of a list. It uses an 
object detection process. Bubble sort algorithm arranges the regions (large to small) to make sure that there is 
at least one big region (target object) in object detection process. It performs a series of passes over the 
sequence: 

a. Compare each element (not including the last one) with its neighbour to the right-if they are out of 
order, swap them. Once the largest element is reached, it keeps on swapping until it goes to the last 
position. 

b. Compare each element (not including the last two) with its neighbour to the right-if they are out of 
order, swap them. Once the second largest element is reached, it keeps on swapping until it goes to the 
second last position. This process continues until there are no unsorted elements are left. 
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2.3 Mathematical Modelling 

The main objective of object tracking is to estimate the state trajectories of an object a moving or 
movable object. It is well-known that a point moving in our 2D space can be described by its 2D position and 
velocity vectors. 


X(K) = [Xk Ve Xk Vir Ox]! (2) 


For instance, “Equation (2)” can be used as a state vector of such a point in the Cartesian coordinate system, 
where ' denotes matrix transpose, x, and y, are the position coordinates along x, y axes, respectively, and 
x, and y;, are the velocity vectors, and w,, is the angular velocity. The system used in this work is modelled 
using the “constant velocity” (CV) model, or more precisely “nearly-constant-velocity model”. The Process 
state and dynamic measurement models are assumed to be linear in discrete time. It can be written as, 


Xk+1 = AX, + Wx (3) 
Zk = Hx, + Vk (4) 


where x; is the N X 1 vector state at discrete time k , z, is the M X 1 vector measurement at discrete-time 
k, H is the constant measurement matrix of size M x N, A is the constant state transition matrix of size M x 
N, w, is an N X 1 zero mean Gaussian distributed processing(or system) noise vector at time k having 
constant covariance matrix of Q, and vy, isan N xX 1 zero mean random measurement noise vector at time 
k having a constant positive definite covariance matrix of R. vz, w; are assumed to be independent of each 
other for all k and j. It is assumed that the period between each observation is a constant A, seconds. 


2.4 Tracking Algorithms 
2.4.1 a-B filtering 
The a-B tracking [29] algorithm can be expressed as follows. Prediction step (time update): 


Xka1 = Xi + AS, (5) 


na 


Sk41 = Sx (6) 


where the vector %, and $= X, are the predicted position matrix and predicted velocity matrix, 
respectively. Correction step (measurement update): 


Re = K+ AZ - XK) (7) 


4a 


Sx. = + i x — X) (8) 


where the vector %; , $, are the estimated position matrix and estimated velocity matrix, respectively, a and 
B are tuning constants between number 0 and number | to smooth the position and velocity estimates, 
respectively. 


2.4.2 Linear Kalman Filter 

The LKF is a time-domain recursive filter with the capability to estimate the state of a dynamic 
system by using a series of measurements. Considering the system model equations “Equation (3)” and 
“Equation (4)”; the Linear Kalman filter equations are given by [30], 


Pe = APL AT +, (9) 

K, = Po H™(HPCHT +R)? (10) 
SAR (11) 
RE = Re +K,(%, — HX) (12) 
Pt = (1-K,H)P;, (13) 
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for k = 1,2,..., where I is the identity matrix. X; is the a priori estimate of the state x, given measurements 
up to and including time k — 1. X; is the a posteriori estimate of the state x, specified measurements up to 
and including time k. K, is the Kalman gain, P, is the covariance of the a priori estimation error (x, — 
R,), and Pg is the covariance of the a posteriori estimation error (x, — %¢). The Kalman filter is 
initialized by: 


RJ = E(x) (14) 
Poy = El (xo — %9 )(%o — %0 )"] (15) 
where E‘(.) is the expectation operator. 


2.4.3. Extended Kalman filter 
The EKF [31] is the nonlinear version of the Kalman filter. The nonlinear models for EKF are 
given by, 


Process Model: x,44, = f(X,) + Wx (16) 
Measurement Model: z, = h(x,) + vp (17) 


The algorithm for the extended Kalman filtering is essentially similar to that of Linear Kalman filtering, 
except following modifications, 


of | ~ 

Fea = = 18h (18) 
oh | a 

H, _ Ox | & (19) 


3. EXPERIMENTS 
3.1. Implementation details 

In this work, we implemented our tracker in MATLAB R2013a PC machine with Intel 17-3770 CPU 
(3.4GHz) with 2GB memory, which runs 29fps on this platform. In addition, the self-made video consisting 
of JPEG image sequence with 720 x 1280 pixels per frame is taken using the iPhone6 camera. The 
environment for the bad light condition is created by switching off all the lights in the room (improper 
illumination), while the environment is considered as a normal light condition when the room is properly 
illuminated. 


3.1.1 Experimental Setup 

The distance considered for tracking the target object is approximately 6meters with target objects 
being balls of different sizes (small, medium and big) whose radii are 3.325cms, 4.9cms and 8.5cms, 
respectively. The video captured using iPhone6 as following specifications: Image Type: RGB colour space, 
Image Dimension: 720 x 1280 pixels (Height= 1280 and Width= 720), Frame Rate: 29 fps, Number of 
frames: For Normal Light condition (For small size ball =54 frames, For medium size ball= 60 frames, For 
big size ball=58frames) and For Bad Light condition (Only for small size ball=70 frames). Table 1 shows the 
initial values of the basic parameters for the tracking algorithms. 


3.2 Performance Measures 
a. Absolute error (AE): AE is the magnitude of the difference between the true value and the tracked value 
of the object. 


€= |Xtrue — Xtrackeal (20) 


where Xtrue the true value of object parameters and Xtrackeq iS the tracked value of the object parameters. 

b. Root Mean Square Error(RMSE): RMSE is one of the most widely used full-reference quality assessment 
metric, which is computed by square root of the average of squared intensity differences between tracked 
(Xtracked) and true image pixels (Xtrye)- 


Indonesian J Elec Eng & Comp Sci, Vol. 10, No. 3, June 2018 : 989 — 999 


Indonesian J Elec Eng & Comp Sci ISSN: 2502-4752 0 993 





4. 


RMSE = NM san Se Cteacked = Xtrue) Z (21) 


where N and M are the image dimensions. 


c. Peak Signal- to- Noise Ratio (PSNR): The value of Peak signal-to-noise ratio can easily be obtained by 
using the mean squared error. 


2 
PSNR = 10 * logig — (22) 
Mean square error: 


ih 
MSE = NM yet wi Seared — Xtrue) 2 23) 


where, L is the dynamic range of the image (8 bits per pixel gray scale images L = 255). 
d. Tracking detection rate (TDR): Tracking detection rate is the ratio of a number of frames in which the 
object is detected to the total number of frames in which the object present. 


Object detected in number of frames 
TR= ——oerreelesq as +100 (24) 


Object present in number of frames 


e. Object tracking error (OTE): Object tracking error is the normal inconsistency in the centroid of the 
tracked object from its true value. It is given by, 





N 
one Xtrue—Xtracked)* -( Ytrue—Ytracked)” 


OTE = (25) 


where, Xtrye and Virye are the actual 2D coordinates of the object, Xtackeq 20d Ytrackeq are the tracked 2D 
coordinates of the object. 


Table 1. Simulation parameters for Trackers 








a-B LKF EKF 

Parameters Values Parameters Values Parameters Values 

a 0.9 dt 1 dt 1 
10d 0 10 dt 0 
010 dt 010 dt 
B 0.005 A 0010 A 001 0 
0001 0001 
1000 1000 
Ax 1 A I 10 4 A F 10 4 
0.1524 0.0143 R 0.2845 0.0045 
ch _ R 0.055 0.0055 0.0045 0.0045 
oe ae Q 0.01 * eye (4) Q 0.01 *eye (4) 
es oa P 100*eye (4) P 100 *eye (4) 





3.3. Experimental results 
3.3.1 Tracking results of small size ball in normal light condition 

The RMSEs of the LKF, a-f filter and EKF for small size ball dataset are given in Table 2. As 
observed in Table 2, the average value of the RMSE was reduced after applying the LKF to small ball data. 
On an average, the minimum RMSE reduction was 0.0330 and the maximum was 0.4478 for small size ball 
data. This result indicates that the predicted output of the LKF is closer to the desired output. The best 
average values are emphasized in Table 2. Similarly, the PSNRs of the LKF, a-f filter and EKF for small 
size ball dataset are given in Table 2. As observed in Table 2, the average value of the PSNR was increased 
after applying the LKF to small ball data. On an average, the maximum PSNR was 48.44dB and the 
minimum was 33.10dB for small size ball data. Figure! shows the error of centroid, error of radius, RMSE, 
centroid and radius of x-y coordinates, AE and OTE in each frame for small size ball based on LKF under 
normal light conditions. Figure 1 shows the error of centroid, error of radius, RMSE, centroid and radius of 
x-y coordinates, AE and OTE in each frame for small size ball based on LKF under normal light conditions. 
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The OTEs of the LKF, a-f filter and EKF for small size ball dataset are given in Table 2. As observed in 
Table 2, the average value of the OTE was reduced after applying the LKF to small ball data. On an average, 
the minimum OTE reduction was 0.0476 and the maximum was 1.3562 for small size ball data. This result 
indicates that the predicted output of the LKF is closer to the desired output. Similarly, the AEs of the LKF, 
a-B filter and EKF for small size ball dataset are given in Table 2. As observed in Table 2, the average value 
of the AE was reduced after applying the LKF to small ball data. On an average, the minimum AE was 
0.1276 and the maximum was 0.9898 for small size ball data. 


Error of kalman filter centroid Vs Frame Number Error of kalman filter radius Vs Frame Number Root Mean Square error 
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Figure 1. The experimental results of small ball tracking using LKF 


3.3.2 Tracking results of medium size ball in normal light condition 

The OTEs of the LKF, a-B filter and EKF for medium size ball dataset are given in Table 2. As 
observed in Table 2, the average value of the OTE was reduced after applying the LKF to medium size ball 
data. On an average, the minimum OTE reduction was 0.0591 and the maximum was 0.6972 for medium size 
ball data. The best average values are emphasized in Table 2. Similarly, the AEs of the LKF, a-f filter and 
EKF for medium size ball dataset are given in Table 2. As observed in Table 2, the average value of the AE 
was reduced after applying the LKF to small ball data. On an average, the minimum AE was 0.2146 and the 
maximum was 1.2519 for small size ball data. This result indicates that the predicted output of the LKF is 
closer to the desired output. 


Table 2. Average parameters comparison for different size of balls using five trackers under 
normal and bad light conditions 








In Normal light condition In bad light condition 
aaa For small size ball a ae SMS For big size ball For small size ball 
a-B LKF EKF  o-B  LKF EKF a-B LKF EKF_  oa-B LKF  EKF 
AE - 0.12 0.76 oe 0.21 1.25 1.31 063 1.34 045 0.11 ~ 1.05 
0.4 0.2 
RMSE 4 0.03 0.17 8 0.04 0.28 O45 0.21 030 0.09 0.02 0.17 
TDR (%) Me 100 100 = 100. 91.6 = = 93.1 100 93.1 85.7 100 85.7 
OTE a 0.04 «0.16 - 0.05 029 1.01 048 0.29 0.22 0.02 0.14 
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The RMSEs of the LKF, a-f filter and EKF for medium size ball dataset are given in Table 2. On an 
average, the minimum RMSE reduction was 0.0489 and the maximum was 0.2869 for medium size ball data. 
Similarly, the PSNRs of the LKF, a-f filter and EKF for medium size ball dataset are given in Table 2. As 
observed in Table 2, the average value of the PSNR was increased after applying the LKF to medium ball 
data. On an average, the maximum PSNR was 45.16dB and the minimum was 28.82dB for medium size ball 
data. Figure 2 shows the error of centroid, error of radius, RMSE, centroid and radius of x-y coordinates, AE 
and OTE in each frame for medium size ball based on LKF under normal light conditions. 


Error of kalman filter centroid Vs frame index Error of kalman filter radius Vs frame index Root Mean Square error 
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Figure 2. The experimental results of medium size ball tracking using LKF 


3.3.3. Tracking results of big size ball in normal light condition 

The OTEs of the LKF, a-B filter and EKF for big size ball dataset are given in Table 2. On an 
average, the minimum OTE reduction was 0.2997 and the maximum was 1.0102 for big size ball data. 
Similarly, the AEs of the LKF, a-f filter and EKF for big size ball dataset are given in Table 2. As observed 
in Table 2, the average value of the AE was reduced after applying the LKF to small ball data. On an 
average, the minimum AE was 0.63999 and the maximum was 1.3412 for big size ball data. 

The RMSEs of the LKF, a-fB filter and EKF for big size ball dataset are given in Table 2. On an 
average, the minimum RMSE reduction was 0.2112 and the maximum was 0.4545 for big size ball data. The 
best average values are emphasized in Table 2. Similarly, the PSNRs of the LKF, a-f filter and EKF for big 
size ball dataset are given in Table 2. As observed in Table 2, the average value of the PSNR was increased 
after applying the LKF to big ball data. On an average, the maximum PSNR was 45.00dB and the minimum 
was 27.91dB for big size ball data. Figure 3 shows the error of centroid, error of radius, RMSE, centroid and 
radius of x-y coordinates, AE and OTE in each frame for big size ball based on LKF under normal light 
conditions. 


3.3.4 Tracking results of small size ball in bad light condition 

The OTEs of the LKF, a-B filter and EKF for small size ball dataset in bad light condition are given 
in Table 2. As observed in Table 2, the average value of the OTE was reduced after applying the LKF to 
small ball data. On an average, the minimum OTE reduction was 0.0227 and the maximum was 0.2275 for 
small size ball data. This result indicates that the predicted output of the LKF is closer to the desired output. 
The best average values are emphasized in Table 2. Similarly, the AEs of the LKF, a-f filter and EKF for 
small size ball dataset are given in Table 2. As observed in Table 2, the average value of the AE was reduced 
after applying the LKF to small ball data. On average, the minimum AE was 0.1194 and the maximum was 
1.0528 for small size ball data. 

The RMSEs of the LKF, a-f filter and EKF for small size ball dataset in bad light condition are 


given in Table 2. As observed in Table 2. On an average, the minimum RMSE reduction was 0.0218 and the 
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maximum was 0.1767 for small size ball data. Similarly, the PSNRs of the LKF, a-B filter and EKF for small 
size ball dataset are given in Table 2. As observed in Table 2, the average value of the PSNR was increased 
after applying the LKF to small ball data. On an average, the maximum PSNR was 45.17dB and the 
minimum was 27.89dB for small size ball data. 


Error of kalman filter centroid Vs frame index Error of kalman filter radius Vs frame index Root Mean Square error 
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Figure 3. The experimental results of big size ball tracking using LKF 


3.3.5 Object tracking frame results of small size ball in normal light condition 

Figure 4 shows some of the test results for small size ball under normal light condition. We have 
evaluated our approach using self-made Database, with the frame size being 720 x 1280. We have selected 
some frames from the video while tracking the object continuously during its movement. The target object is 
marked by the bounding box (red circle). Figure 4 shows 4 frames out of the tracking results based on LKF in 
Figure 4(a), a-B filter in Figure 4(b) and EKF in Figure 4(c). In frames 20 and 30 shown in Figure 4(a), the 
target object tracked using LKF is dominated by the red bounding box as the camera is static and the size of 
the object is reduced with distance. The same phenomena is observed in frames 30, 45 and 20, 26 shown in 
Figure 4(b) and Figure 4(c), using a-f filter and EKF, respectively. Even though the target object is small, the 
proposed tracking algorithms can detect the object (small ball). 


. . { pe = j 5 . 
e e . e e 


\ | j \ \ \ | artbzommensl : \ 


(a) (b) (c) 


Figure 4. Object tracking result of the small size ball: (a) based on LKF (7, 9, 20 and 30 frames), (b) based 
on a-B filter (8, 15, 30 and 45 frames) and (c) based on EKF (7, 10, 20 and 26 frames) 


3.3.6 Tracking frame results of medium size ball in normal light condition 

The Figure 5 shows each 4 frames out of the tracking results based on LKF in Figure 5(a), a- filter 
in Figure 5(b) and EKF in Figure 5(c). In spite of the reflections from the target object on the tiles of the floor 
which is consider as noise, the LKF, a-f filter, and EKF algorithms are effective in tracking the object. 
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Figure 5. Object tracking result of the medium size ball: (a) based on LKF (10, 15, 35 and 50 frames), (b) 
based on a-f filter (9, 12, 34and 52 frames) and (c) based on EKF (8, 18, 38and 48 frames) 





3.3.7 Tracking frame results of big size ballin normal light condition 

The test results for big size ball under normal light condition are shown in Figure 6. We have 
selected some frames from the video while tracking the object continuously during its movement, where the 
target object is marked by a bounding box (red circle). Figure 6 shows each four frames out of the tracking 
results based on LKF in Figure 6(a), o-f filter in Figure 6(b) and EKF in Figure 6(c). 

The values of covariance matrices Q and R plays an important role in the estimation of LKF and 
EKF parameters. In this paper, the values of Q and R were determined using trial error which forms the 
delicacy of the proposed method. Choosing incorrect values of Q and R may lead to increase in RMSE and 
decrease in the accuracy. 


(a) (b) (c) 


Figure 6. Object tracking result of the big size ball: (a) based on LKF (15, 25, 45and 54frames), (b) based on 
a-B filter (16, 28, 40and 52 frames) and (c) based on EKF (18, 26, 48 and 53frames) 


3.3.8 Tracking frame results of small size ball in bad light condition 

The results for small size ball under bad light condition are shown in Figure 7. The target object is 
marked by the bounding box (red circle). Figure 7 shows 4frames out of the tracking results based on LKF in 
Figure 7(a), a-B filter in Figure 7(b) and EKF in Figure 7(c). In frames 43 and 50 shown in Figure 7(a), the 
target object tracked using LKF is dominated by the red bounding box as the camera is static and the size of 
the object is reduced with distance. The same phenomena is observed in frames 34, 48 and 45, 50 shown in 
Figure 7(b) and Figure 7(c), using a-f filter and EKF, respectively. 


(a) (b) (c) 


Figure 7. Object tracking result of the small size ball under bad light condition: (a) based on LKF (25, 35, 43, 
and 50 frames), (b) based on a-f filter (19, 24, 34 and 48 frames) and (c) based on EKF (26, 36, 45, and 
50 frames) 





4. CONCLUSION 

In this paper, a new method is developed for tracking of different sizes of balls using a-B, LKF and 
EKF. In proposed tracking framework, three different tracking algorithms were used to track the target object 
under the normal light condition for different sizes of balls and also under bad light conditions for the small 
size of the ball. Experimental results demonstrate that LKF algorithm is more effective and efficient than a-B 
and EKF algorithms for linear system model applications. In spite of the reflections from the target object on 
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the tiles of the floor which is considered as noise, the LKF, a-f filter, and EKF algorithms are effective in 
tracking the object. For future work, an optimization method could be considered in determining the 
covariance matrices Q and R, to obtain an optimal LKF and EKF parameter estimator more accurately. 
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